We have to constrain Mccabe like this because it's done by Flake8:

https://github.com/PyCQA/flake8/blob/4.0.1/setup.cfg#L42

So, until Flake8 releases a new version, we can't change that constrain here, sorry.

We have to constrain Mccabe like this because it's done by Flake8:

https://github.com/PyCQA/flake8/blob/4.0.1/setup.cfg#L42

So, until Flake8 releases a new version, we can't change that constrain here, sorry.

Alright, I'll keep this open then until it is ready to merge.

Note that flake8 version 4.0.1 does actually work just fine with mccabe-0.7.0, the main branch already reflects this as of January this year. In Gentoo we already have that patch for flake8, and also this patch for python-lsp-server, both test suites pass without problems.

May I ask what the rational is behind pinning a dependency because another dependency pins it? I noticed this happens more often in the spyder stack, and it makes maintaining this in Gentoo a bit difficult. I get that sometimes things need to be pinned because of incompatibilities, but my life would be a lot easier if we wouldn't have these "indirect" dependencies/pins. See also:

• https://bugs.gentoo.org/791313
• https://bugs.gentoo.org/843251
• https://bugs.gentoo.org/816132
• https://bugs.gentoo.org/815160
• https://bugs.gentoo.org/743466
• https://bugs.gentoo.org/803269
• https://bugs.gentoo.org/855095
• https://bugs.gentoo.org/783621

Note that flake8 version 4.0.1 does actually work just fine with mccabe-0.7.0, the main branch already reflects this as of PyCQA/flake8#1542 this year.

I knew that already.

May I ask what the rational is behind pinning a dependency because another dependency pins it?

I can't say about Flake8, but in our case we have to follow its pins or it could fail to load due to not having the right dependencies.

I noticed this happens more often in the spyder stack, and it makes maintaining this in Gentoo a bit difficult.

I understand but that's because rolling Linux distros want to provide the latest versions of everything without considering possible incompatibilities with newer versions or that maintainers don't always have time to update their requirements. However, that represents a fringe case among the general population of users (perhaps ~20% among the ~5% of Linux users in total, so a 1%?)

Sorry to be blunt, but we prefer to be cautious with our dependencies instead of trying to support everyone. Mac and Windows users (and most Linux users, I'd say) don't care about using the latest version of every package. They just want to use a stable program that don't break often when new dependencies are released. After years of experience we've learned that the simplest and most reliable way to do that is by adding pins.

I get that sometimes things need to be pinned because of incompatibilities, but my life would be a lot easier if we wouldn't have these "indirect" dependencies/pins

That's not going to change, sorry.

May I ask what the rational is behind pinning a dependency because another dependency pins it?

I can't say about Flake8, but in our case we have to follow its pins or it could fail to load due to not having the right dependencies.

Isn't it the package managers responsibility to pull in the correct dependencies of the dependencies? I.e. if the installed version of flake8 requires a specific version of mccabe then the package manager (pip or whatever) should install that version, even if a different package allows a wider range of mccabe versions.

I noticed this happens more often in the spyder stack, and it makes maintaining this in Gentoo a bit difficult.

I understand but that's because rolling Linux distros want to provide the latest versions of everything without considering possible incompatibilities with newer versions or that maintainers don't always have time to update their requirements. However, that represents a fringe case among the general population of users (perhaps ~20% among the ~5% of Linux users in total, so a 1%?)

This is not my philosophy, nor is it Gentoo's. I did not bring this up because of some "outdated==bad" dogma. I don't think there is anything wrong with keeping older versions in the tree for compatibility reasons if necessary. The reason I bring this up is because of annoying blockers that are a result of unnecessary pinning. I.e. package X depends on foobar<1 and package Y depends on foobar>1, and all of a sudden I can't install package X and Y simultaneously any more.

Here is what happens time and time again: A package (in this case flake8) at some point releases a new version (i.e. 4.0.2) compatible with mccabe-0.7, now I cannot install this version because python-lsp-server still requests mccabe<0.7, I have to wait until the next release of python-lsp-server. This is only a minor annoyance until some other package starts requiring mccabe>=0.7, now all of a sudden I have a conflict and I cannot install both packages at the same time. This is a real problem that happens surprisingly often, and people (rightfully) complain to me about it. It has gotten a lot better though since I started sed'ing out many of the indirect dependencies so the package manager can properly sort it out.

Sorry to be blunt, but we prefer to be cautious with our dependencies instead of trying to support everyone. Mac and Windows users (and most Linux users, I'd say) don't care about using the latest version of every package. They just want to use a stable program that don't break often when new dependencies are released. After years of experience we've learned that the simplest and most reliable way to do that is by adding pins.

As I said, I don't bring it up because I care so much about using the latest version, I bring it up because of the blockers. I want things to "just work" just as much as the next user. Which is exactly the point, it doesn't "just work" when you have to deal with blockers as a result of unnecessary pinning. And this is not just a Gentoo problem, nor a Linux problem for that matter. I get the exact same blockers on Windows (I do occasionally (have to) use python/spyder on a Windows system). The only difference being that on Windows I can't easily test if the dependency can be unpinned, and if so, fix the blocker.

Anyway, tl;dr, by all means pin things when there is an (expected) incompatibility, but I humbly request to keep it to a minimum and not pin things solely because something else pins it. It saves me and others work, and it means less blockers on the user end.

That's not going to change, sorry.

😞

The reason I bring this up is because of annoying blockers that are a result of unnecessary pinning. I.e. package X depends on foobar<1 and package Y depends on foobar>1, and all of a sudden I can't install package X and Y simultaneously any more.

Ok, I see. Then I think Spyder and its dependencies are not well suited for Linux distros because one of the nice things about it is that it can be installed in a virtualenv and use Python packages from another env to run code. So, we're not expecting that Spyder works alongside every other Python package under the sun. Just that it doesn't break when its dependencies are updated and that works with a small set of packages from the Scientific Python ecosystem (e.g. Numpy, Pandas and Matplotlib).

Furthermore, I believe Spyder should be distributed as similar applications (e.g. VSCode or RStudio), i.e. as a single, self-contained bundle because trying to force it to work with newer versions of our dependencies usually generates errors or segfaults.

As I said, I don't bring it up because I care so much about using the latest version, I bring it up because of the blockers. I want things to "just work" just as much as the next user. Which is exactly the point, it doesn't "just work" when you have to deal with blockers as a result of unnecessary pinning.

That's the way Linux distros in general handle packages and versions, i.e. in a single tree that forces them to constantly fix the blockers you mentioned (perhaps the exception is NixOS, not really sure). But that's a problem generated by the packaging model they use, not a problem with Spyder or how we handle dependencies.

In the world Spyder is used the most (i.e. Conda first and then Pip), you can create separate virtualenvs and avoid those blockers quite easily. In addition, not using pins is a source of lot of headaches for us because people can install any Spyder version they want in a virtualenv. So if a version doesn't have the right set of constraints, it will end up trying to work with packages that break it in one way or another.

For instance, we didn't constrain qtconsole until Spyder 5.2, but Anaconda decided to package 5.1.5 in its latest version, which is semi-broken with the latest qtconsole. And that increased the number of bugs we receive per month to 400+, when we usually receive around 200.

Anyway, tl;dr, by all means pin things when there is an (expected) incompatibility

The problem is we don't know when that's going to happen. So better (for us, of course) to be extra-safe than deal with frustrated users. At the end, I think the most important problem is how your software is used, not how it's packaged (that's a second-order problem, I'd say).

😞

Sorry, we can't please everybody. And our Conda + Windows/Mac users are much more than our Linux + rolling distros ones.

Closing in favor of #257.